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DETECTION AND SEGMENTATION OF SWEEPS IN 
COLOR GRAPHICS IMAGES 

Background of the Invention 

[0001] The present invention relates to color graphics image 

processing. It finds particular application in conjunction with detection and 
segmentation of sweeps in color graphics images, and will be described with particular 
5 reference thereto. However, it is to be appreciated that the present invention is also 
amenable to other like applications. 

[0002] Content-based image classification has emerged as an 

important area in multimedia computing due to the rapid development of digital imaging, 
storage, and networking technologies. A reproduction system, such as a copier or a 
10 printer, strives for a pleasing rendering of color documents. Picture/graphics classifiers 
have been developed to differentiate between a picture image and a graphics image with 
high accuracy by analyzing low-level image statistics. 

[0003 ] For example, U. S . Patent No . 5 , 767,978 to Revankar et 

al. discloses an adaptable image segmentation system for differentially rendering black 
15 and white and/or color images using a plurality of imaging techniques. An image is 
segmented according to classes of regions that may be rendered according to the same 
imaging techniques. Image regions may be rendered according to a three-class system 
(such as traditional text, graphic, and picture systems), or according to more than three 
image classes. In addition, only two image classes may be required to render high 
20 quality draft or final output images. The image characteristics that may be rendered 
differently from class to class may include half toning, colorization and other image 
attributes. 

[0004] A color output device such as a CRT computer monitor, 

liquid crystal display, inkjet printer, xerographic printer, etc can display a limited range 
25 of colors (the gamut of the output device). If the colors in an image do not reside 



wholly with in an output device gamut, a gamut-mapping algorithm is often applied to 
map the image colors to colors that the output device can produce. A simple approach 
is to preserve in-gamut colors without alteration and clip out-of-gamut colors to the 
closest in-gamut color. More sophisticated techniques can be used. Ideally, the gamut- 
5 mapping algorithm used should be tailored to the image type. For example, a region of 
smoothly varying colors should appear smoothly varying on the output device. Were 
the colors of a sweep to exceed the gamut of an output device, the aforementioned 
clipping approach will show disagreeable artifacts. In fact, it may be desirable to 
sacrifice color fidelity within the gamut to achieve a smooth color transition. Thus 

10 knowing that a region is, or contains, a sweep aids in color reproduction. In general, 
coloring schemes (gamut-mapping algorithms) are tailored for specific types of images 
to obtain quality reproduction. Once an image has been identified as a graphics image, 
further identification of image characteristics can be used to fine-tune the coloring 
schemes for more appealing reproductions. The most prominent characteristics of a 

1 5 graphics image include patches or areas of the image with uniform color and areas with 
uniformly changing colors. This invention focuses on the identification of the second 
characteristic. 

[0005] One example where areas with uniformly changing color 

can usually be observed is in the gradient backgrounds of color business presentation 

20 slides. These areas of uniformly changing color are called sweeps and are constructed 
in the three-dimensional color space as a line during the construction of the synthetic 
graphics. A sweep is constructed by a mathematical formula to cause adjacent pixels to 
change color in a smooth, predicable way. For example, one can use linear interpolation 
of two colors specified for the sweep and render the original image by plotting pixels of 

25 interpolated colors such that neighboring spatial regions are rendered with colors from 
neighboring color regions. One can contemplate other mathematical descriptions of 
curves that achieve like effects. If such a document is printed or scanned, the sweeps 
do not exactly contain the colors on the line due to halftone noise introduced. If a 
reproduction system can correctly identify and segment the sweep areas in an image, the 

30 original sweeps can be reconstructed in the color space and rendered. The sweeps thus 
rendered will be very smooth and the noise introduced by the halftone will not be 
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reproduced. Secondly, if the extreme colors of the sweep can be automatically 
identified, the coloring schemes can be tailored to maximize the smoothness as well as 
contrast and differentiation among colors to render business graphics documents. 

[0006] Further identification of the properties of the graphics 

5 image can be used to fine-tune the coloring scheme to obtain a more appealing 
reproduction. The detection of sweeps in a graphics image can be used to reconstruct 
synthetic sweeps that may otherwise be perturbed due to half toning, scanning artifacts, 
or aging of a document or for other reasons. The extent of the sweeps (i.e., the change 
from color 1 to color 2) may also be used to tailor the coloring scheme to achieve best 
10 smoothness, contrast and differentiation among colors in the reconstructed sweeps. 

[0007] The present invention proposes a new and improved 

method for detecting and segmenting sweeps in a color graphics image that overcomes 
the above-referenced problems and others. 



15 Summary of the Invention 

[0008] In accordance with one aspect of the present invention, 

sweeps in a graphics image are detected and segmented. The input image is transformed 
into an appropriate color space (e.g., CEELUV) and sweep segment information from 
one or more color channel histograms of the image is detected. Then the graphics image 
20 is segmented into sweep and non-sweep areas using the sweep segment information. 

[0009] Still further advantages and benefits of the present 

invention will become apparent to those of ordinary skill in the art upon reading and 
understanding the following detailed description of the preferred embodiments. 



25 Brief Description of the Drawings 

[0010] The invention may take form in various components and 

arrangements of components, and in various steps and arrangements of steps. The 
drawings are only for purposes of illustrating preferred embodiments and are not to be 
construed as limiting the invention. 
3 0 [001 1 ] FIG. 1 is a flowchart of the process to detect and segment 

sweeps in a graphics image in accordance with the present invention; 
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[0012] FIG. 2 shows an embodiment for sub-process 106 of FIG. 

1; 

[0013] FIG. 3 shows an embodiment for sub-process 108 of FIG. 

1; 

5 [0014] FIG. 4 shows an embodiment for sub-process 110 of FIG. 

1; 

[0015] FIG. 5 shows an embodiment for sub-process 1 12 of FIG. 

1; and 

[0016] FIG. 6 shows an embodiment for optional sub-process 

10 114 of FIG. 1. 

Detailed Description of the Preferred Embodiments 

[0017] With reference to FIG. 1, a flowchart of the process to 

detect and segment sweeps in a graphics image 100 is provided. The process 100 begins 

15 with an input graphics image 102. Next, the input graphics image is transformed into 
a three-dimensional histogram in an appropriate color space 104. In the preferred 
embodiment, the CIELUV color space is used. This space is chosen because additive 
mixtures of two colors 1 and 2 lie along a locus that is close to a straight line in three- 
dimensional CIELUV space. It also provides rough perceptual uniformity, i.e., the 

20 Euclidean distance between a pair of colors specified in CIELUV roughly corresponds 
to perceived color difference. Other color spaces with similar properties such as 
CIEL*a*b* are contemplated in this invention. Color sweeps appear as lines in the 
three-dimensional CIELUV color histogram. However, since further processing (e.g., 
edge detection, line detection, etc.) in the three-dimensional space is difficult and time 

25 consuming, the present invention operates on two-dimensional projections of the three- 
dimensional histogram. Lines project as either lines or points in any two-dimensional 
projection from a three-dimensional space. Continuing with the flowchart of FIG. 1, 
two-dimensional histograms are estimated from the three-dimensional histogram 106. 
Next, the two-dimensional histograms are processed to detect and segment sweeps 108. 

30 Sweep segment information from the processing of the two-dimensional histograms is 
combined 110. The combined sweep segment information is used to process the input 
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graphics image to identify and segment sweeps 1 12. Post-processing may be optionally 
and selectively used to reject false alarms (i.e., areas falsely identified as sweeps) 114. 
The process ends 116 after post-processing 1 14 or, if post-processing is not used, after 
the input graphics image is processed 112. 
5 [0018] FIG. 1 depicts an embodiment of the method of the 

present invention, comprising: 1) converting an input graphics image to a color space 
104; 2) projecting the image to a number of planes within the color space 106; 3) 
detecting curves in each plane and linking overlapping curves 108; 4) associating pixels 
with colors that project to the detected curves in each plane with evidence that the color 

10 belongs to a sweep 108; and 5) for each pixel color, combining sweep evidence to 
determine whether pixels of that color are part of a sweep 110, 112. A preferred 
embodiment of the method of FIG. 1 uses the CEELUV color space 104, projects the 
image to three orthogonal planes 106, detects curves using a Hough transform and edge 
linking 108, identifies whether pixels are associated with colors of the detected curves 

15 with a logical label (e.g., TRUE/FALSE) 108, and uses a logical AND operation to 
combine sweep evidence 110, 112. 

[0019] With reference to FIG. 2, a preferred embodiment for 

estimating two-dimensional histograms from the three-dimensional histogram 106 is 
provided. In the preferred embodiment, the first step of the sub process estimates the 

20 two-dimensional histograms (size: GL x GL, GL is typically 256) from the input image 
in UV, LU, and LV projections, respectively 206. Next, the histograms are normalized 
by a scaling scheme to create H_UV, H_LU, and H_LV histograms, in UV, LU, and LV 
projections respectively 208. For example, in a scaling scheme the bin values are 
integers between 0 and GL-1 (typically 255). The normalized histograms are considered 

25 as gray-scale images for further processing. 

[0020] With reference to FIG. 3, the preferred embodiment for 

processing the two-dimensional histograms to detect sweep segments 108 is provided. 
In the preferred embodiment, the first step of the sub process segments the H_UV, 
HJLU, and H LV histograms by detecting edges and creating an edge map 308 for each 

30 projection. A standard edge detector, such as the commonly known Canny edge 
detector (sigma = 5, mask size = 51, lower threshold = 0.4, higher threshold = 0.9), can 



-5- 



be used for detecting edges in the HUV, HLU, and H_LV histograms. The 
parameters identified for the edge detector were determined empirically. By using a 
larger mask size, thick edges are detected but small edges are missed. By using a smaller 
mask size, thin edges are detected but thick edges are missed. In experiments, the mask 
5 size of 5 1 obtained the desired balance between detecting thin and thick edges that 
correspond to sweeps in histogram images. Next, a connectivity analysis of the edges 
is performed on each of the H UV, H LU, and H LV edge maps 310. For example, 
a standard 8-connected component algorithm can be used to perform the connectivity 
analysis and ignore very small edges (e.g., less than 30 pixels). Note that each sweep 

10 appears as a line segment in the histogram images and is detected as two parallel edges 
in the edge map. The edge maps are binary images with white pixels representing edges 
and black pixels representing non-edges. 

[0021] With continuing reference to FIG. 3, in order to estimate 

the location and orientation of the sweep line segments, the detected edges in the H_UV, 

1 5 H_LU, and H_LV edge maps are converted to points in a Hough parametric space 3 12. 
For example, a standard Hough Transform algorithm can be used to convert the edges 
in the spatial domain to the Hough parametric space. Points in the Hough space with 
a large number of votes (i.e., more than a threshold T) are selected. Experimentally, a 
satisfactory threshold T can be empirically determined so that most of the lines longer 

20 than 20 pixels are detected. The selected points represent straight lines in the H_UV, 
H_LU, and H_LV histograms. The lines detected in the Hough space are in parametric 
form and run to infinity at both ends. However, identification of end points of sweeps 
is important. 

[0022] For this purpose, each detected line in the Hough space 

25 is rendered (e.g., drawn as a 3-pixel wide line) on the edge map using a standard scan- 
line-drawing algorithm 3 14. Typically, the rendering is performed using a particular gray 
value (e.g., 100) and the overlap between the edges and the line drawn are marked as 
"overlap." The extremities of the pixels marked as "overlap" are also noted. These 
extremities define the line segments in the two-dimensional histogram images. As 
30 mentioned before, each sweep in the image appears as parallel line segments in the edge 
map. Hence, the pairs of parallel line segments in the edge maps are identified 316 and 
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are considered for farther processing while other segments are ignored. Each pair of 
parallel line segments correspond to a single sweep in the original image. Next, the mid 
segment of each pair of parallel line segments is computed 318 and recognized as a 
sweep. 

5 [0023] The sweeps detected as segments in the HUV, H LU, 

and HLV edge maps are projections of the original sweeps from the three-dimensional 
color space. Segment information consists of a data structure that indicates for each pair 
(u, v), (1, u) and (1, v) whether or not the pair corresponds to a sweep. To segment the 
input graphics image into sweep regions and non-sweep regions, the sweep segment 

10 information from the three projections must be combined. Referring to FIG. 4, a 
preferred embodiment for combining sweep segment information from the two- 
dimensional histograms 110 is provided. In this embodiment, the sweep segment 
information from the UV, LU, and LV projections are combined 410. A restrictive 
combination scheme performs an "and" operation on the information from the three 

15 projections. Specifically, the "and" condition states that a pixel with color (1, u, v) is in 
a sweep if and only if , from the three projections, (u, v) is a sweep AND (1, u) is a 
sweep AND (1, v) is a SWEEP. Alternatively, a liberal scheme performs an "or" 
operation on the information from the three projections: a pixel with color (1, u, v) is in 
a sweep if and only if, from the three projections, (u, v) is a sweep OR (1, u) is a sweep 

20 OR (1, v) is a SWEEP. The choice between restrictive and liberal combination schemes 
depends on the application. This invention contemplates other logical or functional 
combinations that may give advantage for other applications. 

[0024] Once the sweep segments in each of the H_UV, H_LU, 

and H LV edge maps are identified and noted and the UV, LU, and LV projections are 

25 combined, the original graphics image is revisited. Referring to FIG. 5, a preferred 
embodiment for processing the input graphic image to identify sweep segments 1 12 is 
provided. In this embodiment, each pixel in the input graphics image is labeled either 
"sweep" or "non-sweep." The sub process begins with selection of a first pixel 512. 
Next, the distance between the sweep segments and the pixel is computed 514 and 

30 compared to a particular threshold value 516. If the distance is less than the threshold, 
the pixel is labeled "sweep" 518. Otherwise, the pixel is labeled "non-sweep" 520. 
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Finally, the sub process determines if it has reached the last pixel 522. Until the last 
pixel is reached, the sub process is repeated for the next pixel. When the last pixel is 
reached, the sub process is complete. 

[0025] For synthetic graphics this scheme for detecting and 

5 segmenting sweeps works well. However, for scanned graphics images, the result from 
this segmentation scheme may have an unacceptable number of errors. Often, for a 
liberal scheme, there are far more false alarms than false misses. A post-processing stage 
may be used to reject several types of false alarms. Referring to FIG. 6, several potential 
components of optional post-processing 114 are provided. The optional components 

10 may be implemented individually or in any combination. First, the post-processing 
scheme may use a digital filter to check for connectivity and reject small isolated areas 
of sweeps and non-sweeps 614. For example, a median digital filter was used in 
experiments. Second, the post-processing scheme may compute the gradient 
information in the image and reject those areas where the gradient in the image is less 

1 5 than a threshold 6 1 6. Third, a more sophisticated type of gradient post-processing could 
check for consistency of the gradient at several scales 618. Finally, the post-processing 
scheme may ignore gray sweeps by rejecting the detected horizontal lines in the H_LU 
and HJLV edge maps 620. 

[0026] Although the Hough transform is used to detect straight 

20 line segments in this embodiment, known variants of the Hough methodology can be 
used to detect parameterized curves, surfaces, and shapes. Other color spaces and 
sweeps may produce other curves or surfaces in the three projections and these can be 
detected with methods known in the art and are within the scope of this invention. 

[0027] The invention has been described with reference to the 

25 preferred embodiments. Obviously, modifications and alterations will occur to others 
upon reading and understanding the preceding detailed description. It is intended that 
the invention be construed as including all such modifications and alterations insofar as 
they come within the scope of the appended claims or the equivalents thereof. 



